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This is in response to the appeal brief filed 12/07/2004. 
(1) Real Party in Interest 
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A statement identifying the real party in interest is contained in the brief. 

(2) Related Appeals and Interferences 

A statement identifying the related appeals and interferences which will directly affect or 
be directly affected by or have a bearing on the decision in the pending appeal is contained in the 
brief. 

(3) Status of Claims 

•The statement of the status of the claims contained in the brief is correct. 

(4) Status of Amendments After Final 

The appellant's statement of the status of amendments after final rejection contained in 
the brief is correct. 

(5) Summary of Invention 

The summary of invention contained in the brief is deficient because Appellant's 
arguments or allegations cannot be construed as contents in the summary of invention. The 
Examiner considers these arguments as misleading to the patentability of the claimed invention. 

(6) Issues 

The appellant's statement of the issues in the brief is correct. 

( 8) Claims Appealed 

The copy of the appealed claims contained in the Appendix to the brief is correct. 

(9) Prior Art of Record 

6,240,423 Hirata 05-2001 

(10) Grounds of Rejection 

The following ground(s) of rejection are applicable to the appealed claims: 
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Claims 1-7, 11-12, 16-19 are rejected under 35 U.S.C. 102(e). This rejection is set forth 
in prior Final Office Action, dated 06/04/2004, and the prior Advisory Action, 1 1/16/2004. 

1. Claim 1: 

Hirata teaches a method of visualizing and retrieving a data file comprising: 
Displaying a plurality of images representing corresponding data files on a display device 
using a first distance metric between each data file ( e.g.. between each data file for the plurality 
of images including the query image wherein either the similarity measure is involved or the 
mutual similarity measure is involved : Calculating a first distance metric between a query image 
and images in the database using a region-based and/or boundary-based matching to produce a 
first set of similar images. Images being retrieved in order based upon the distance between the 
query image and the result candidate images, column 6. Subsequently, based on the mutual 
similarities between the images, retrieval results of the candidate images are sorted. The images 
similar to each other are assumed to be under one group and are re-ordered based on the 
similarity. The users may specify the forms of the output for display; figures 1-11; column 6-14. 
Note that Fig. 10A in relation to Fig. 8 shows displaying a plurality of images retrieved using the 
first distan ce metric wherein the keyword "display" can be found in column 6. lines 10-25 that 
the one-dimensional display of the query results o f the images have been displayed in Fig. 10 A 
and 8 ): 

Redisplaying a portion of the images on the display device using a refined distance metric 
(e.g., Hirata teaches using a second distance metric such as the distance metric used for 
grouping and re-ordering based on similarity among result candidates to retrieve images based 
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on the set of similar images retrieved using a first distance measure. The second distance metric 
is now a "refined" distance metric because it refines the search for images. The second distance 
metric of Hirata thus meets the claim limitation of "a refined distance metric. " Here, a refined 
set of similar images is produced from the first set of similar images produced using a first 
distance metric. See figure 9, 11 and 12; column 6-14. Moreover, based on the mutual 
similarities between the candidate images, retrieval results of the candidate images are sorted 
and displayed in a two-dimensional way in Fig. 10B. The images similar to each other are 
assumed to be under one group and are re-ordered based on the similarity and users can specify 
the forms of the output for display; figures 1-11; column 6-9. Note that Fig. 10B shows 
redisplaying a plurality of images in a two-dimensional way using a refined distance metric in 
accordance with the distance measure among the candidate images. The distance measure has 
been illustrated in Fig. 9 and re-order based on the similarity and the distance measure takes 
place so that the images are grouped in terms of similarities among them and similarities among 
the candidates are calculated to form groups and the order in which the search results are 
presented is changed based on the grouping. The term "redisplaying" means that Fig. 10 A has 
displayed a set of search results of the images in a one-dimensional array and Fig. 10B re- 
displayed a set of search results arranged in two rows with the appropriate grouping wherein 
the first row of the images in Fig. 10B is less than the first row of the candidate images in Fig. 
10 A and the second row of the images in Fig. 10B is less than the total number of the candidate 
images in Fig. 10A. "A portion of the images" merely means either 100% or less than 100% of 
the plurality of the images retrieved in terms of the number of images. "A refined distance 
metric " merely means a distance metric that refines the search process for the images or a 
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distance metric that imposes additional conditions than the first distance metric used in the 
"displaying" step)\ and 

Performing at least one of retrieving, marking, and selecting at least one desired data file 
{Hirata teaches at least one of clicking on one of the displayed results of the retrieved 
hyper linked images to select one desired data file for the hyper linked image to display the image 
in a full window, e.g., a popup window, and the displayed image information are retrieved from 
at least one desired data file at user *s selection of the image. Hirata further discloses retrieving 
at least one desired data file at the step of displaying or at the step of redisplaying; see figures 1- 
11; column 6-14). 

In other words, Hirata is directed to outputting the candidate images (the second set of 
similar images) based on the first distance metric (e.g., stored in a memory) and displaying the 
candidate images in a one-dimensional way (column 6). Hirata further discloses redisplaying a 
portion of displayed images using a refined distance metric in a two-dimensional way (column 
6). Hirata teaches in column 7 that the step of grouping and re-ordering based on similarity 
among result candidates. 

Moreover, in column 6-14 or in the claims 37-39, Hirata clearly teaches displaying the 
second set of similar images grouped by the clusters on the display and DISPLAYING RE- 
ORDERED SECOND SET of SIMILAR IMAGES ON THE DISPLAY and therefore at least 
two steps of displaying is involved in the process. 

In claim 23 of Hirata as enabled by column 6-9 n of Hirata, the first set of similar images 
are produced by a first similarity between a query image and the images using region-based 



Application/Control Number: 09/904,627 Page 7 

Art Unit: 2672 

image matching and second set of similar images are produced based on a second similarity 
between the query image and the first set of similar images using boundary based image 
matching. In claim 24 of Hirata, the second set of similar images is grouped into clusters based 
upon a similarity calculation among members of the second set of similar images. In claim 37, 
the second set of images grouped by the clusters is displayed on the display . Hirata thus teaches 
displaying the second set of similar images representing data files (as images retrieved from data 
files) on a display device (displaying involves a display device) using a second similarity (based 
on the first set of similar images retrieved by the first similarity) between each data file of the 
data files for images including the query image. Thus, Hirata meets the claim limitation of 
"displaying a plurality of images representing corresponding data files on a display device using 
a first distance metric between each data file". 

In claim 25 of Hirata, the second set of similar images is re-ordered based upon a 
similarity calculation among members of the second set of similar images. In claim 38 of Hirata. 
the re-ordered second set of images is displayed on the display, thereby resulting in a re- 
displaying . Hirata thus teaches redisplaying the re-ordered second set of images (which is 100% 
or less than 100% of the second set of images first displayed as set forth in the claim 37 of 
Hirata) on the display using a mutual similarity calculation among members of the second set of 
similar images (which is a refined distance metric). This step has been set forth in the claim 38 of 
Hirata. Thus, Hirata meets the claim limitation of "redisplaying a portion of the images on the 
display device using a refined distance metric". 

In claim 39 of Hirata, a representative image from each of the clusters is selected and 
whereupon for each cluster the selected representative image is displayed (redisplayed) on the 
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display. Hirata thus teaches redisplaying a strictly less than 100% of the similar images produced 
in the displaying step. By the selection of representative image being displayed, Hirata also 
discloses performing a selection of the representative image as retrieved from a data file and 
therefore Hirata teaches the claim limitation of performing at least one of retrieving, marking, 
and selecting at least one desired data file. 

From the embodiment of Hirata's teaching, Hirata disclosess in column 6 a similary 
matching among the candidate images resulting from the region-based and boundary-based 
matching. Not only users can select regions for the image matching or querying, but also users 
can specify other parameters such as the threshold values to determine how the images are 
categoried into groups and this categorization is used to REDUCE the number of images to be 
displayed, e.g., in a particular row of the display. 

Hirata discloses in column 7 that the number of results (the number of the input images) 
to be processed in the step of grouping and re-ordering based on similarity among the second set 
of similar images can be CONTROLLED BY THE USER, and such a selection further limits the 
number of the images to be re-displayed. 

The displaying and re-displaying of the images are interative because users may limit the 
second set of similar images by selecting the threshold values for the clustering of the second set 
of similar images either before retrieving or during retrieving by the application program, 
thereby determining how the portion of the second set of similar images are selected on the 
display during the two-steps of displaying process. 

By selecting the number of the candidate images in the step of re-ordering based on the 
mutual similarity and the selection of the representative image in each cluster, the resulting 
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images being redisplayed is strictly less than the second set of similar images retrieved in the 
displaying step. Hirata thus teaches redisplaying a portion (say 100% or less than 100%) of 
displayed images selected in the previous step of the image matching using a mutual similarity 
measure. Users' selection of the candidate images produces a further refined set of the candidate 
images in the performing step. Finally, it is noted that the mutual similarity as calculated in 
Hirata's column 9, lines 1-9 meets the claim limitation of a distance measure. 

From the teaching of column 6-7 and 14, it is concluded that Hirata at least discloses two 
separate steps of displaying the candidate images wherein a lesser number of the clustered 
candidate images can be re-ordered and re-displayed. Hirata further teaches interactive user 
selection for selecting a subset of images to be re-ordered and re-displayed based on the mutual 
similarity and user selection of parameters for retrieval and the represenative images in each 
cluster. 

Claim 2: 

The claim 2 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of repeating the redisplaying step until a desired data file is identifiable. 
However, Hirata further discloses the claimed limitation of repeating the redisplaying step until a 
desired data file is identifiable (e.g., column 4 wherein Hirata teaches specifying a hierarchical 
structure using the authoring tools in which a user may repeat the step of specifying regions of 
the query image until a desired data file is identifiable). 



Claim 3: 
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The claim 3 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of computing a feature vector for each data file and calculating the first 
distance metric between each data file using a first subset of data contained in the feature vector. 
However, Hirata further discloses the claimed limitation of computing a feature vector for each 
data file (e.g., similarity is determined based on a number of features derived from the respective 
images; column 1) and calculating the first distance metric between each data file using a first 
subset of data contained in the feature vector (e.g., column 6-9). 

Claim 4: 

The claim 4 encompasses the same scope of invention as that of claim 3 except additional 
claimed limitation of calculating a second distance metric between each data file using a second 
subset of data contained in the feature vector which is greater than the first subset. However, 
Hirata further discloses the claimed limitation of calculating a second distance metric between 
each data file using a second subset of data contained in the feature vector which is greater than 
the first subset (e.g., column 6-9). 

Claim 5: 

The claim 5 encompasses the same scope of invention as that of claim 4 except additional 
claimed limitation of computing the feature vector for each data file before starting the method; 
storing the feature vector for each data file; and accessing the feature vector for each data file. 
However, Hirata further discloses the claimed limitation of computing the feature vector for each 
data file before starting the method; storing the feature vector for each data file; and accessing 
the feature vector for each data file (e.g., grouping the second set of similar images into clusters 
based upon a similarity calculation among members of the second set of similar images and re- 
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ordering the second set of similar images based upon a similarity calculation among members of 
the second set of similar images wherein feature vector calculation is involved for each data file 
and the results are stored in the database; see column 9). 
Claim 6: 

The claim 6 encompasses the same scope of invention as that of claim 4 except additional 
claimed limitation of each feature vector having a length at least eight. 

However, Hirata further discloses the claimed limitation of each feature vector having a length at 
least eight (e.g., considering the multiple features with respect to the image regions). 
Claim 7: 

The claim 7 encompasses the same scope of invention as that of claim 4 except additional 
claimed limitation of the feature vector including at least one of a color feature and a texture 
feature. However, Hirata further discloses the claimed limitation of the feature vector including 
at least one of a color feature and a texture feature (e.g., figures 1-11; column 2). 

Claim 11: 

The claim 1 1 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of the data files being image files. However, Hirata further 
discloses the claimed limitation of the data files being image files (e.g., figures 1-11; column 5). 

Claim 12: 

The claim 12 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of the data files being video files. However, Hirata further discloses 
the claimed limitation of the data files being video files (e.g., figures 1-11; column 5). 



Application/Control Number: 09/904,627 Page 12 

Art Unit: 2672 

Claim 16: 

The claim 16 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of the portion of the images redisplayed being graphically selected 
by the user. However, Hirata further discloses the claimed limitation of the portion of the images 
redisplayed being graphically selected by the user (e.g., figures 1-11; column 4). 

2. Claim 17: 

Hirata teaches a method of interactively retrieving a data file from a set of data files in 
real time comprising: 

Displaying a plurality of images, each image corresponding to a data file, on a display 
device using a first distance metric between each data file (e.g., Based on the mutual similarities 
between the images, retrieval results of the candidate images are sorted. The images similar to 
each other are assumed to be within one group and are re-ordered based on the similarity and 
users can specify the forms of the output for display. Clustering image data files is done using a 
distance metric such as a first similarity; figures 1-11; column 6-9); 

Interactively selecting, by a user, a portion of the images (e.g., a user specify the regions 
of the images; e.g., the region division of figure 3; figures 1-11; column 6-9); 

Redisplaying the portion of the images in real time on the display device using a refined 
distance metric (e.g., Hirata teaches using a distance metric such as a second similarity to 
retrieve images based on the set of similar images retrieved using a first similarity. The second 
similarity is now a "refined" distance metric because it refines the search for images. The second 
similarity of Hirata thus meets the claim limitation of "a refined distance metric." Here, a refined 
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set of similar images is produced from the first set of similar images produced using a first 
distance metric. Re-order is based on the similarity among the candidate images of figure 9 or 2 nd 
stage image matching is based on boundary of figure 1 1 and grouping and re-ordering is based 
on the similarity among candidates using the refined distance metric of figure 11; figures 1-11; 
column 4); and 

Retrieving a desired data file (retrieved data file is displayed on a display device; figures 
1-11; column 4). 

Claim 18: 

The claim 18 encompasses the same scope of invention as that of claim 17 except 
additional claimed limitation of computing a feature vector for each data file and calculating the 
first distance metric between each data file using a first subset of data contained in the feature 
vector. However, Hirata further discloses the claimed limitation of computing a feature vector 
(visual attributes values, including color, shape and texture-related values; see column 2) for 
each data file and calculating the first distance metric between each data file using a first subset 
of data contained in the feature vector (a subset of data refers to the image region and Hirata 
teaches extracting the feature vector corresponding to each image region in the feature-based 
image processing; figures 1-11; column 6-9). 

Claim 19: 

The claim 19 encompasses the same scope of invention as that of claim 18 except 
additional claimed limitation of calculating a second distance metric between each data file using 
a second subset of data contained in the feature vector which is greater than the first subset. 
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However, Hirata further discloses the claimed limitation of calculating a second distance metric 
between each data file using a second subset of data contained in the feature vector which is 
greater than the first subset (e.g., re-ordering the output images using a second distance metric; 
figures 1-11; column 6-9). 



Claims 8, and 13-14 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hirata et al. U.S. Patent No. 6,240,423 (hereinafter Hirata), as applied to claims 1 and 17 above, 
and further in view of Moghaddam et al. U.S. Patent No. 6,584,221 (hereinafter Moghaddam). 
This rejection is set forth in prior Final Office Action, dated 06/04/2004, and the prior Advisory 
Action, 11/16/2004. 

Claim 8: 

(1) The claim 8 encompasses the same scope of invention as that of claim 4 except 
additional claimed limitation of the feature vector including at least one of a color histogram, 
color moment, color coherence histogram, Multiresolution Simultaneous Augoregressive 
(MRSAR) Model, coarseness, and directionality. 

(2) Hirata is silent on the claimed limitation of the feature vector including at least one of 
a color histogram, color moment, color coherence histogram, Multiresolution Simultaneous 
Augoregressive (MRSAR) Model, coarseness, and directionality. 

(3) Moghaddam discloses the claimed limitation of the feature vector including at least 
one of a color histogram, color moment, color coherence histogram, Multiresolution 
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Simultaneous Augoregressive (MRSAR) Model, coarseness, and directionality (e.g., 
Moghaddam column 3, lines 2-10; column 3, lines 64-67; column 4, lines 1-8). 

(4) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Moghaddam' s feature vectors into Hirata's method of visualizing and retrieving an image from 
the database because Hirata suggests visual and semantic features including color, shape and 
texture-related values (Hirata column 2). 

(5) One having the ordinary skill in the art would have been motivated to do this because 
it would have advantageously provided the various image features for accurate image extraction 
and matching. 

Claims 13-14: 

(1) The claim 13 encompasses the same scope of invention as that of claim 1 except 
additional claimed limitation of establishing a fixed scale that spans a maximum distance 
between the plurality of data files; and indicating a relative position on the fixed scale for the 
redisplay of the portion of the image, thereby providing the user with a reference frame. 
The claim 14 encompasses the same scope of invention as that of claim 1 except additional 
claimed limitation of the fixed scale being at least one of a linear scale, a logarithmic scale, and a 
hyperbolic scale. 

(2) Hirata is silent on the linear scale. 

(3) Moghaddam discloses the linear scale (e.g., Moghaddam column 4). 
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(4) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Moghaddam's linear scale into Hirata's method of visualizing and retrieving an image from the 
database because Hirata suggests specifying image regions (Hirata column 2). 

(5) One having the ordinary skill in the art would have been motivated to do this because 
it would have advantageously selected image regions at different resolutions. 

Claims 9, 10, and 20 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Hirata et al. U.S. Patent No. 6,240,423 (hereinafter Hirata), as applied to claims 1 and 17 above, 
and further in view of Jain U.S. Patent No. 6,121,969 (hereinafter Jain). This rejection is set 
forth in prior Final Office Action, dated 06/04/2004, and the prior Advisory Action, 1 1/16/2004. 

3. Claims 9, 10, and 20: 

(1) Hirata teaches a method of visualizing and retrieving a data file from a set of data 

files. 

(2) Hirata is silent on the claimed limitation that the first distance metrics are mapped 
into an N-dimensional space using FastMap for displaying and the refined distance metrics are 
mapped into an N-dimensional space using FastMap for redisplaying. 

(3) Jain discloses the claimed limitation that the first distance metrics are mapped into an 
N-dimensional space using FastMap for displaying and the refined distance metrics are mapped 
into an N-dimensional space using FastMap for redisplaying (e.g., Jain column 27, lines 30-40; 
column 25, lines 35-67; column 26, lines 1-57). 
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(4) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Jain's mapping method into Hirata's method of visualizing and retrieving an image from the 
database because Hirata suggests distance metric for image features with respect to image 
regions (Hirata column 4-5). 

(5) One having the ordinary skill in the art would have been motivated to do this because 
it would have advantageously provided the display of metric properties for representation in the 
display space (Jain column 25, lines 35-67; column 26, lines 1-57). 

Claim 15 is rejected under 35 U.S.C. 103(a) as being unpatentable over Hirata et al. U.S. 
Patent No. 6,240,423 (hereinafter Hirata) as applied to claims 1 and 17 above, and further in 
view of Bates et al. U.S. Patent No. 5,528,259 (hereinafter Bates). This rejection is set forth in 
prior Final Office Action, dated 06/04/2004, and the prior Advisory Action, 1 1/16/2004. 

Claim 15: 

(1) Hirata teaches a method of visualizing and retrieving a data file from a set of data 

files. 

(2) Hirata is silent on the claimed limitation of providing a display depth indication that 
represents an amount of overlapping of images on the display and scrolling to view images that 
were previously not viewable due to overlapping of the images. 

(3) Bates discloses the claimed limitation of providing a display depth indication that 
represents an amount of overlapping of images on the display and scrolling to view images that 
were previously not viewable due to overlapping of the images (e.g., Bates figures 4A-4C; 
column 4, lines 39-56; column 5, lines 25-45). 
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(4) It would have been obvious to one of ordinary skill in the art to have incorporated the 
Bates' scrolling of overlapping images and selecting of an alternate layer into Hirata's method of 
visualizing and retrieving an image from the database because Hirata suggests a user interface 
selection of a portion of image and second level regions and objects (Hirata figures 1-11; column 
4). 

(5) One having the ordinary skill in the art would have been motivated to do this because 
it would have advantageously provided a pictorial representation of one of selected multiple 
layers of image data in a multi-dimensional scrolling method of manipulating or selecting image 
layers (e.g., Bates figures 4A-4C; column 4, lines 39-56; column 5, lines 25-45). 

(11) Response to Argument 

On Page 4 and 10 in the remarks, the Appellant argued with respect to the claim 1 and similar 
claims in substance: 

(A) "Independent claims 1 and 17 recite, among other features, redisplaying a portion of 
the images on the display device using a refined distance metric. In contrast, the Hirata patent is 
directed to a query based image matching system, wherein images are displayed only a single 
time per query." 

In response to the arguments in (A), Hirata at least discloses two steps of displaying the 
image results, i.e., displaying the second set of similar images from query and displaying 
(redisplaying) the re-ordered set of the second set of similar images using a different distance 
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metric such as the similarity measure together with the mutual distance calculations among the 
second set of similar images. 

Moreover, in column 6-14 or in the claims 37-39, Hirata clearly teaches displaying the 
second set of similar images grouped by the clusters on the display and REDISPLAYING RE- 
ORDERED SECOND SET of IMAGES ON THE DISPLAY and therefore at least two steps of 
displaying is involved in the process. 

In column 6-14 and more specifically, the claim 37 of Hirata, the second set of similar 
images grouped by the clusters is displayed on the display. Hirata thus teaches displaying the 
second set of similar images representing data files (as images retrieved from data files) on a 
display device (displaying involves a display device) using a second similarity (based on the first 
set of similar images retrieved by the first similarity) between each data file of the data files for 
images including the query image. In column 6-14 and more specifically the claim 38 of Hirata, 
the re-ordered second set of similar images is displayed (i.e., re-displayed) on the display. Hirata 
thus teaches redisplaying the re-ordered second set of images (which is 100% or less than 100% 
of the second set of images first displayed as set forth in the claim 37 of Hirata) on the display 
using a mutual similarity calculation among members of the second set of similar images (which 
is a refined distance metric). Hirata also teaches redisplaying a strictly less than 100% of the 
second set of similar images in the re-displaying step because Hirata discloses in column 6-14 as 
set forth in claim 39 that, a representative image from each of the clusters is selected and 
whereupon for each cluster the selected representative image is displayed (redisplayed) on the 
display, i.e., only a portion of the images being re-displayed on the display. 
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On Page 6 in the remarks, the Appellant argued with respect to the claim 1 and similar claims in 
substance: 

(B) "The present application discloses a multi-media database and classification system, 
which can, for example, provide for automatic classification and retrieval of multimedia files 
based on features on the multimedia files. ..As described on specification page 6 beginning with 
line 4, the distance metric can be refined at each of plural redisplays until a desired data file is 
found or a maximum refined distance metric is reached." 

In response to the arguments in (B), Appellant's arguments that "the distance metric can 
be refined at each of plural redisplays until a desired data file is found or a maximum refined 
distance metric is reached" cannot be found in the claim 1. Appellant's claim 1 is broadly 
construed rather than incorporating the specific argument in the claim. 

On Page 7 in the remarks, the Appellant argued with respect to the claim 1 and similar claims in 
substance: 

(C) ". . .Based on perceived properties of a desired image, a user can select an area 202 of 
the screen where a desired image most likely resides. Figure 2B shows a portion of images 
redisplayed as selected by area 202. Here the distance metric has been recalculated using more of 
the image information than was used in the first distance calculation." 

In response to the arguments in (C), Appellant's arguments contains a term of "an area 
202 of the screen" which differs from the claim limitation of "a portion of images". A portion of 
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images could be a subset of the plurality of images which refers to the number of images rather 
than the image sub-region or region within a complete image. "A portion of images" could also 
be a plurality of subimages wherein the subimage corresponds to a region or a sub-region of the 
image. "A portion of images" could also refer to 100% of the plurality of images. Appellant 
failed to particularly points out and distinctly claim the subject matter which the Appellant 
regards as invention. 

Appellant further argues that "the distance metric has been recalculated using more of the 
image information than was used in the first distance calculation." However, Hirata discloses 
reordering based on a distance metric together with the mutual similarity and the distance 
calculations among the second set of similar images wherein the distance information among the 
similar images is more of the image information than the distance metric used for retrieving the 
second set of similar images (See column 6-14 of Hirata). 

On Page 8 in the remarks, the Appellant argued with respect to the claim 1 and similar claims in 
substance: 

(D) "The Hirata patent is not directed to redisplaying a portion of displayed images using 
a refined distance metric. In contrast to Appellants' claimed use of a refined distance metric to 
redisplay a portion of displayed images, Hirata teaches applying a new distance metric to all 
images during each image matching operation." 

In response to the arguments in (D), Hirata at least discloses, during each image 
matching operation , two steps of displaying the image results, i.e., displaying the second set of 
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similar images and redisplaying the re-ordered second set of the similar images using a distance 
metric such as the mutual similarity measure together with the distance calculations among the 
second set images. Hirata thus applying a refined distance metric to the second set of similar 
images during the re-ordering step. 

For example, Hirata teaches using a second distance metric to retrieve images based on 
the set of similar images retrieved using a first distance measure. The second distance metric is 
now a "refined" distance metric because it refines the search for images that reorders the 
candidate images. The second distance metric of Hirata thus meets the claim limitation of "a 
refined distance metric." Here, a set of similar images is produced from the set of similar images 
produced using a first distance metric. See figure 9, 1 1 and 12; column 6-9. 

Moreover, based on the mutual similarities between the images, retrieval results of the 
candidate images are sorted. The images similar to each other are assumed to be under one group 
and are re-ordered based on the similarity and users can specify the forms of the output for 
display; figures 1-11; column 6-9. Note that Fig. 10B shows a display (or a redisplay) of a 
plurality of images using a refined distance metric in accordance with the distance measure 
among the candidate images. The distance measure has been illustrated in Fig. 9 and re-order 
based on the similarity distance measure allows for the images to be grouped in terms of the 
mutual similarities and similarities are calculated to form groups. The order in which the search 
results are presented is changed based on the grouping. 

The term "redisplaying" means that Fig. 10A has displayed a set of search results of the 
images in a one-dimensional array and Fig. 10B re-displayed the same set of search results 
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(which are arranged in two rows with the appropriate grouping wherein the first row of the 
images in Fig. 10B is less than the first and only row of the candidate images in Fig. 10A and the 
second row of the images in Fig. 10B is less than the total number of the candidate images in 
Fig. 10A). "A portion of the images" merely means either 100% or strictly less than 100% subset 
of the plurality of the images retrieved. "A refined distance metric" merely means a distance 
metric that refines the search process for the images or a distance metric that imposes more 
restrictive conditions in the redisplaying step than the distance metric used in the "displaying" 
step. 

On Page 9 in the remarks, the Appellant argued with respect to the claim 1 and similar claims in 
substance: 

(E) "In addition, the Hirata patent fails to teach or suggest the claim 17 feature of 
interactive user selection for selecting a subset of images to be displayed. Rather, the query 
based system of the Hirata patent groups all images, and only creates new groupings based on a 
new query. The Hirata patent does not teach or suggest applying a refined distance metric in a 
second stage of image matching to redisplay a portion of displayed images." 

In response to the arguments in (E), Hirata discloses in column 6-14 as set forth in claim 
39, a representative image from each of the clusters is selected and whereupon for each cluster 
the selected representative image is displayed (redisplayed) on the display, i.e., only a portion of 
the images being re-displayed on the display. Hirata also discloses user's selection of the 
parameters by the user or an automatic selection of the parameters by the application program for 
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the image retrieval to limit the number of the second set of similar images. Hirata thus teaches 
interactive user selection for selecting a subset of images to be displayed. 

On Page 10-1 1 in the remarks, the Appellant argued with respect to the claim 8, 13 and 14 in the 
Second groupings in substance: 

(F) "Claims 8, 13 and 14 stand rejected under 35 U.S.C. 103(a) as being unpatentable 
over the Hirata patent, and further in view of U.S. Patent 6,584,221 (Moghaddam et al.), claims 
9, 10 and 20 stand rejected under 25 U.S.C. 102(e) as being unpatentable over the Hirata patent, 
and further in view of U.S. Patent 6,121,969 (Jain et al.); and claim 15 stands rejected under 35 
U.S.C. 103(a) as being unpatentable over the Hirata patent, and further in view of U.S. Patent 
5,528,259 (Bates et al.). For at least these reasons, the Hirata patent does not disclose or suggest 
each and every element of claims 8-10, 13-15 and 20, and the patents to Moghaddam et al., Jain 
et al. and Bates et al. do not make up for the deficiencies of the Hirata patent. Accordingly, the 
rejection should be reversed." 

In response to the arguments in (F), Appellant applies the same arguments for the claim 1 
to Claims 8, 13 and 14 in the Second Grouping and to Claims 9-10 and 20 in the third Grouping 
and to Claim 15 in the fourth Grouping. The responses to arguments (A)-(E) also applies to these 
groupings of the claims. 

On Page 13 in the remarks, the Appellant argued with respect to the claim 1 and similar claims in 
substance: 
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(G) "The image matching based on boundary, as taught by the Hirata patent, does not 
redisplay a portion of the images as claimed, and does not suggest redisplay of a portion of the 
images first displayed. Rather, the first and second similarity calculations disclosed by the Hirata 
patent are a mere sequence of calculations in a method for querying a database." 

In response to the arguments in (G), Hirata at least discloses, during each image 
matching operation , two steps of displaying the image results as set forth in column 6-14 and 
claims 37-39 of Hirata, i.e., displaying the second set of similar images and redisplaying the re- 
ordered set of the second set of similar images using a mutual similarity measure together with 
the distance calculations among the second set of similar images . 

On Page 13 in the remarks, the Appellant argued with respect to the claim 1 and similar claims in 
substance: 

(H) "Moreover, a similarity matching among candidate images resulting from the Hirata 
patent's region-based and boundary-based matching does not produce a redisplay of a portion of 
the already displayed images for retrieving. The Hirata patent does not disclose a first visual 
display of a plurality of images, and redisplaying a portion of the first displayed images, the 
portion being in reference to the first display." 

In response to the arguments in (H), Hirata discloses displaying the second set of similar 
images retrived using the region-based and boundary-based matching. Hirata further discloses re- 
displaying redisplaying the re-ordered set of the second set of similar images using a distance 
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metric such as the mutual similarity measure together with the distance calculations among the 
second set of similar images . 

On Page 13 in the remarks, the Appellant argued with respect to the claim 17 and similar claims 
in substance: 

(I) "The Examiner asserts that the displaying and re-displaying of resulting candidate 
images are interactive because users can select the number of the candidate images. However, 
the selection criteria are not based upon visual inspection of the first display and defining a 
portion of the first display for redisplay." 

In response to the arguments in (I), the argument of "based on visual inspection of the 
first display" cannot be found in the claim 17. In view of Hirata's related teaching, Hirata 
discloses in column 6-14 as set forth in claim 39 that, a representative image from each of the 
clusters is selected and whereupon for each cluster the selected representative image is displayed 
(redisplayed) on the display, i.e., only a portion of the images being re-displayed on the display. 
Hirata further discloses user's selection of the parameters by the user or an automatic selection of 
the parameters by the application program for the image retrieval to limit the number of the 
second set of similar images. The selection criteria can be defined by the user or via the 
application program and therefore a portion of the first display for redisplay can be defined 
interatively by the user or via the application program. Hirata thus teaches interactive user 
selection for selecting a subset of images from the first display by the user through user's 
selection of the parameters or threshold values or via the application program so that the 
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selection and re-ordering of the second set of similar images are based on the user's selection of 
the parameters and re-displaying the re-ordered set of the images. 



For the above reasons, it is believed that the rejections should be sustained. 
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